Y Combinator is open-sourcing an agent harness called QM (short for quartermaster) designed to manage a fleet of agents for startups and YC employees. The system aims to provide flexible, easy-to-administer tools that can be used as personal assistants or assigned to specific projects to handle work-related tasks.
- It is intended to allow every employee and project to have their own agent as needed.
- This follows previous internal experiments with Ruby-based loops and Hermes agents.
- The codebase is available at github.com/yc-software/qm.
- It is intended to allow every employee and project to have their own agent as needed.
- This follows previous internal experiments with Ruby-based loops and Hermes agents.
- The codebase is available at github.com/yc-software/qm.
The QM repository provides a multiplayer agent harness designed specifically for startups, allowing multiple employees to have isolated workspaces while still collaborating via Slack or web interfaces. The system is built with an architecture that separates the core logic from specific model harnesses and deployment configurations, enabling users to switch between various providers like Claude Code or Codex without being tied to a single vendor. It offers tiered security postures—ranging from strict human approval for all tools to high-speed autonomous operation—and supports background work through scheduled crons and webhooks.
- Supports multiple backends including Pi, OpenCode, Codex, and Claude Code.
- Offers three distinct security modes: Strict (human intervention required), Auto (AI-driven screening), and Dangerous (no screening).
- Provides per-scope memory, files, keychain views, permissions, and durable sandboxes for each user or room.
- Includes an "individual auth" feature where users can connect their own AI accounts to keep usage credentials separate from the organization's shared keys.
- Supports multiple backends including Pi, OpenCode, Codex, and Claude Code.
- Offers three distinct security modes: Strict (human intervention required), Auto (AI-driven screening), and Dangerous (no screening).
- Provides per-scope memory, files, keychain views, permissions, and durable sandboxes for each user or room.
- Includes an "individual auth" feature where users can connect their own AI accounts to keep usage credentials separate from the organization's shared keys.
Yuhao Wu writes about HarnessDev, a benchmark that evaluates LLMs' ability to build and iteratively improve their own agent harness—the model-external execution infrastructure that wraps a model and shapes its task performance. The benchmark has two stages: Creation, where the agent builds a complete execution system from a minimal seed and a few cases, and Evolution, where it revises its own harness using downstream execution feedback. Generated harnesses substantially lag behind mature human-engineered references on code and search/research, while matching or exceeding them on writing and machine-learning experimentation, with large variation in execution cost.
- Covers six creator LLMs across four domains and five downstream benchmarks (2,207 unique instances).
- Hidden evaluation tasks are withheld from development to prevent overfitting.
- Evolution gains are unstable and transfer only partially to held-out tasks.
- Performance gains depend strongly on which model executes the harness, indicating limited cross-model transfer.
- Covers six creator LLMs across four domains and five downstream benchmarks (2,207 unique instances).
- Hidden evaluation tasks are withheld from development to prevent overfitting.
- Evolution gains are unstable and transfer only partially to held-out tasks.
- Performance gains depend strongly on which model executes the harness, indicating limited cross-model transfer.
Paolo Perrone states the code "harness" around an LLM agent affects cost more than capability, breaking it into five layers: execution boundary, sandboxing, memory persistence, verification loops, and context pipelines.
The agent loop is six lines of Python; engineering determines what the model does, accesses, retains, checks, and sees, affecting token spend and unsanctioned actions.
A June 2026 preprint measured up to a 40x difference in tokens per solved task across three harnesses on the same model, while pass-rate gaps were 0-8 percentage points with confidence intervals crossing zero.
- The motivating anecdote: a prompt rule forbidding unapproved edits was violated 76 times in one afternoon; rewriting it as a pre-dispatch hook that returns a deny eliminated the failures entirely.
A denylist sandbox is defeated by path traversal (`work/../secrets/api_key`), whereas an allowlist combined with `os.path.normpath` before comparison catches every spelling of the same file.
The author notes six issues: no peer-reviewed harness study by August 2026, harness gains don't generalize to new tasks, a fine-tuning success contradicted its reward signal, harnesses aren't portable across models, the field lacks a definition, and harness effects decrease with better base models.
- All five layers ship as runnable Python scripts in a public repo (github.com/paoloap-py/agent-harness-guide) that use a scripted stand-in for the model, so every failure mode is reproducible without an API key.
The agent loop is six lines of Python; engineering determines what the model does, accesses, retains, checks, and sees, affecting token spend and unsanctioned actions.
A June 2026 preprint measured up to a 40x difference in tokens per solved task across three harnesses on the same model, while pass-rate gaps were 0-8 percentage points with confidence intervals crossing zero.
- The motivating anecdote: a prompt rule forbidding unapproved edits was violated 76 times in one afternoon; rewriting it as a pre-dispatch hook that returns a deny eliminated the failures entirely.
A denylist sandbox is defeated by path traversal (`work/../secrets/api_key`), whereas an allowlist combined with `os.path.normpath` before comparison catches every spelling of the same file.
The author notes six issues: no peer-reviewed harness study by August 2026, harness gains don't generalize to new tasks, a fine-tuning success contradicted its reward signal, harnesses aren't portable across models, the field lacks a definition, and harness effects decrease with better base models.
- All five layers ship as runnable Python scripts in a public repo (github.com/paoloap-py/agent-harness-guide) that use a scripted stand-in for the model, so every failure mode is reproducible without an API key.
This article explores the concept of an "agent harness," the essential software infrastructure that wraps around a Large Language Model (LLM) to enable autonomous, goal-directed behavior. While foundation models provide the core reasoning capabilities, the harness manages the orchestration loop, tool integration, memory, context management, state persistence, and error handling. The author breaks down the eleven critical components of a production-grade harness, drawing insights from industry leaders such as Anthropic, OpenAI, and LangChain. By comparing the harness to an operating system and the LLM to a CPU, the piece provides a technical framework for understanding how to move from simple demos to robust, production-ready AI agents.
AutoAgent is an autonomous framework designed for agent engineering, functioning similarly to autoresearch but focused on building and iterating on agent harnesses. The system allows a user to assign a task to an AI agent, which then autonomously modifies system prompts, tools, agent configurations, and orchestration over time. By running benchmarks and checking scores, the meta-agent performs a hill-climbing optimization, keeping improvements and discarding failures. The core workflow involves programming via a Markdown file called program.md, which provides context and directives to the meta-agent, while the meta-agent directly edits the agent.py harness file. This approach minimizes manual engineering by allowing the agent to optimize its own performance through continuous, automated experimentation.
This article by Sebastian Raschka explores the fundamental architecture of coding agents and agent harnesses. Rather than focusing solely on the raw capabilities of Large Language Models, the author delves into the surrounding software layers—the "harness"—that enable effective software engineering tasks. The piece identifies six critical components: providing live repository context, optimizing prompt shapes for cache reuse, implementing structured tool access, managing context bloat through clipping and summarization, maintaining structured session memory, and utilizing bounded subagents for task delegation. By examining these building blocks, the article illustrates how a well-designed system can significantly enhance the practical utility of both standard and reasoning models in complex coding environments.
